Image Processing Apparatus, Image Processing Method, and Recording Medium

ABSTRACT

An image processing apparatus includes a still/motion determining unit and a motion blur adder. The still/motion determining unit makes a still/motion determination for each region of respective unit images constituting motion image data. The motion blur adder then adds motion blur to the motion image data on the basis of the still/motion determination and imaging information expressing parameters whereby the motion image data was generated as a result of imaging by an imaging apparatus.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus and animage processing method whereby motion blur is added to motion images,as well as to a recording medium storing a program that causes acomputer or other computational apparatus to execute such imageprocessing.

2. Description of the Related Art

Consider a series of motion images imaged by an imaging apparatus havingshutter functions, wherein the effective exposure time of each frame isshort with respect to the period determined by the frame rate. When suchmotion images are displayed using a display device, the motion of movingobjects in the motion images will be displayed in a discontinuousmanner, and in some cases result in a visual reduction in picturequality wherein a viewer viewing the motion images perceives overlappingimages.

Such motion image quality reduction due to unnatural motion is typicallyreferred to as motion jerkiness, and is defined in the ANSIT1.801.02-1996 standard.

A specific example of the production of motion jerkiness will now bedescribed with the use of the motion image imaging apparatus 500 andmotion image playback apparatus 600 shown in FIG. 20.

The motion image imaging apparatus 500 images and encodes motion imagesaccording to a standard such as MPEG (Moving Picture Experts Group), andthen records the encoded image data onto a recording medium 700 such asa DVD (Digital Versatile Disc).

More specifically, the motion image imaging apparatus 500 includesimaging optics 510 that focus light from the object, imaging elements520 that receive and convert the light focused by the imaging optics 510into an image signal, and a coding processor 530 that encodes the imagesignal output by the imaging elements 520.

In addition, the motion image imaging apparatus 500 is also providedwith a transmit processor 540 that externally transmits image dataencoded by the coding processor 530 via a transmission path, and arecording processor 550 that records image data encoded by the codingprocessor 530 onto a DVD or similar recording medium 700.

The imaging optics 510 include an aperture mechanism 511 that adjuststhe amount of incident object light, and an optical lens system 512 thatcauses object light (whose amount has been adjusted by the aperturemechanism 511) to be focused onto the light-accepting surface of theimaging elements 520.

Meanwhile, the motion image playback apparatus 600 decodes encoded imagedata and then outputs the result to a display or similar device.

More specifically, the motion image playback apparatus 600 includes areceive processor 610 that receives encoded image data transmitted viathe transmission path, and a read processor 620 that reads encoded imagedata from the recording medium 700.

In addition, the motion image playback apparatus 600 is also providedwith a decoding processor 630 that decodes encoded image data outputfrom the receive processor 610 and the read processor 620, as well as anoutput unit 640 that outputs an image signal decoded by the decodingprocessor 630 to a display or similar device.

When imaging motion images outdoors in a bright environment, forexample, a motion image imaging apparatus 500 of the related art likethat described above appropriately controls the exposure amount so as tolimit the amount of light incident on the imaging elements 520 bynarrowing the aperture mechanism 511.

Normally, however, the image is blurred by diffraction phenomena if theaperture mechanism 511 is excessively narrowed. For this reason, themotion image imaging apparatus 500 conducts suitable exposure control byincreasing the shutter speed in addition to adjusting the amount oflight by means of the aperture mechanism 511.

Exposure control is also conducted by modifying the shutter speed inmotion image imaging apparatus that are not provided with aperturemechanisms.

By increasing the shutter speed in this way, the motion image imagingapparatus 500 is able to conduct suitable exposure control for outdoorimaging and similar situations.

However, when a motion image imaging apparatus 500 of the related artexcessively increases the shutter speed, a phenomenon occurs wherein theimage motion no longer appears smooth to the human eye.

Such picture quality reduction due to unnatural motion is typicallyreferred to as motion jerkiness. In other words, if the motion imageimaging apparatus 500 acquires motion images such that jerkiness isproduced, then the motion image playback apparatus 600 will decode anddisplay the jerky image data as-is on the output unit 640.

SUMMARY OF THE INVENTION

As described above, images imaged using an extremely fast shutter speedwith respect to the motion image frame rate are highly sharp whendisplayed as still images, but when displayed as a series of motionimages, the motion of objects in the motion images is not smooth andlooks unnatural to the human eye.

Consequently, the applicant has previously proposed technology wherebymotion vectors are detected for motion images, and motion blur is thenappropriately added to the motion images using the motion vectors (seeJapanese Unexamined Patent Application Publication No. 2007-274299,equivalent to EP 2003876/A2).

However, significant processing load is incurred to detect motionvectors for respective regions in each frame constituting a series ofmotion images, for example, and in some cases such processing may beunsuited to simple apparatus or systems.

It is thus desirable to provide an image processing apparatus, an imageprocessing method, and a recording medium whereby a simple techniquewith a light processing load is used to appropriately add motion blur tomotion images and reduce jerkiness.

An image processing apparatus in accordance with an embodiment of thepresent invention includes: still/motion determining means for making astill/motion determination for each region of respective unit imagesconstituting motion image data; and motion blur adding means for addingmotion blur to the motion image data on the basis of the still/motiondetermination and imaging information expressing parameters whereby themotion image data was generated as a result of imaging by an imagingapparatus.

The motion blur adding means may also add motion blur to regions of therespective unit images determined to be motion regions by thestill/motion determining means.

The still/motion determining means may also generate per-region scorevalues on the basis of the imaging information and the result of adetermination whether a given region is a still region or a motionregion. The motion blur adding means may then add motion blur torespective unit images of the motion image data on the basis of thescore values.

The still/motion determining means may also generate the score values bymeans of weighted addition using plural types of imaging information.

The motion blur adding means may also add motion blur such that theamount of motion blur increases with faster shutter speeds, as specifiedby shutter speed information included in the imaging information.

The motion blur adding means may also add motion blur such that theamount of motion blur increases with increased aperture values, asspecified by aperture value information included in the imaginginformation.

The motion blur adding means may also add motion blur such that theamount of motion blur increases with larger focal lengths, as specifiedby focal length information included in the imaging information.

The motion blur adding means may also add motion blur such that theamount of motion blur increases with increased gyroscope rotation, asspecified by gyroscope rotation information included in the imaginginformation.

The motion blur adding means may include: a reduced image generatorconfigured to generate, from the unit images, one or more reduced imageshaving different resolutions; and a pixel compositing processorconfigured to extract, from the one or more reduced images, pixelspositionally corresponding to a subject pixel in the unit images, and,on the basis of the still/motion determination and the imaginginformation, add motion blur to the unit images by conducting weightedaddition of the subject pixel and the one or more extracted pixels thatwere extracted from the one or more reduced images.

The image processing apparatus may also be provided with imaging meansfor imaging an object to generate the motion image data. The motionimage data obtained by the imaging means may then be input into thestill/motion determining means and the motion blur adding means.

The image processing apparatus may also be provided with playback meansfor playing back motion image data by means of recording medium playbackoperations. The motion image data played back by the playback means maythen be input into the still/motion determining means and the motionblur adding means.

The image processing apparatus may also be provided with receiving meansfor receiving motion image data. The motion image data received by thereceiving means may then be input into the still/motion determiningmeans and the motion blur adding means.

An image processing method in accordance with another embodiment of thepresent invention includes the steps of: making a still/motiondetermination for each region of respective unit images constitutingmotion image data; and adding motion blur to the motion image data onthe basis of the still/motion determination and imaging informationexpressing parameters whereby the motion image data was generated byimaging.

A recording medium in accordance with another embodiment of the presentinvention stores a program that causes a computer to execute the abovestill/motion determining step and motion blur adding step.

In an embodiment of the present invention, processing is conducted toadd motion blur to unit images (image data for single frames, forexample) in input motion image data. The motion blur is added on thebasis of per-region still/motion determination, as well as imaginginformation expressing parameters whereby the unit images were imaged.The still/motion determination is processing to determine, on aper-region basis, whether a given region in a frame (i.e., a pixel unitor pixel block unit made up of a plurality of pixels) is a motion regionor a still region. It is possible to conduct the still/motiondetermination using simple processing, such as by comparison withadjacent frames, for example.

Imaging information refers to information such as shutter speedinformation, aperture value information, focal length information, andgyroscope rotation information (e.g., information indicating horizontaland vertical motion of the imaging apparatus), for example.

On the basis of the above still/motion determination and imaginginformation, motion blur is added to regions in the motion images wherejerkiness is likely to occur. In so doing, motion blur can be addedwithout the heavy processing load incurred by detecting motion vectorsfor each region within a frame.

Thus, according to an embodiment of the present invention, suitablemotion blur can be added to individual unit images in a series of motionimages, in accordance with still/motion determination and imaginginformation expressing parameters whereby the motion images were imaged.As a result, motion images with reduced jerkiness that appears morenatural to the human eye is output.

More particularly, suitable motion blur can be simply added without theheavy processing load associated with methods such as motion vectordetection, thereby enabling simplification of the apparatus andreduction in costs.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an image processing apparatus in accordancewith the basic configuration of an embodiment of the present invention;

FIG. 2 is a block diagram of a motion image imaging apparatus inaccordance with a first embodiment of the present invention;

FIG. 3A is a diagram for explaining imaging information in an embodimentof the present invention;

FIG. 3B is a diagram for explaining imaging information in an embodimentof the present invention;

FIG. 4A is a diagram for explaining imaging information in an embodimentof the present invention;

FIG. 4B is a diagram for explaining imaging information in an embodimentof the present invention;

FIG. 4C is a diagram for explaining imaging information in an embodimentof the present invention;

FIG. 5 is a block diagram of a motion region determining unit inaccordance with an embodiment of the present invention;

FIG. 6 is a block diagram of a motion blur adder in accordance with anembodiment of the present invention;

FIG. 7 is a diagram for explaining weighted addition using reducedimages in accordance with an embodiment of the present invention;

FIG. 8 is a diagram for explaining processing to generate reduced imagesin accordance with an embodiment of the present invention;

FIG. 9 is a diagram for explaining processing to generate horizontallyreduced images in accordance with an embodiment of the presentinvention;

FIG. 10 is a diagram for explaining processing to generate verticallyreduced images in accordance with an embodiment of the presentinvention;

FIG. 11 is a diagram for explaining processing conducted by a reducedimage generator in accordance with an embodiment of the presentinvention;

FIG. 12 is a generalized diagram for explaining processing conducted bya reduced image generator in accordance with an embodiment of thepresent invention;

FIG. 13 is a diagram for explaining a filter in a reduced imagegenerator in accordance with an embodiment of the present invention;

FIG. 14 is a diagram for explaining a reduced image generator inaccordance with an embodiment of the present invention;

FIG. 15 is a diagram for explaining processing conducted by a motionblur adder in accordance with an embodiment of the present invention;

FIG. 16 is a block diagram of the configuration of a motion blur adderin accordance with an embodiment of the present invention;

FIG. 17 is a diagram for explaining pixel compositing and pixelweighting in accordance with an embodiment of the present invention;

FIG. 18 is a diagram for explaining a weighting information table inaccordance with an embodiment of the present invention;

FIG. 19 is a block diagram of a playback and receiving apparatus inaccordance with an embodiment of the present invention; and

FIG. 20 is a block diagram of a motion image imaging apparatus and amotion image playback apparatus of the related art.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, preferred embodiments for carrying out the presentinvention will be described in detail and with reference to theaccompanying drawings. An image processing apparatus in accordance withan embodiment of the present invention adds motion blur to jerky motionimages that appear unnatural to the human eye, thereby reducing suchjerkiness.

The description hereinafter will proceed as follows.

1. Basic Configuration Example 2. First Embodiment: Motion Image ImagingApparatus

[2-1: General Configuration of Motion Image Imaging Apparatus]

[2-2: Imaging Information and Score Value Generation]

[2-3: Exemplary Configuration and Operation of motion Blur Adder]

[2-4: Summary and Conclusions]

[3. Second Embodiment: Playback and Receiving Apparatus] [4. RecordingMedium] [1. Basic Configuration Example]

First, a basic configuration example of an image processing apparatus inaccordance with an embodiment of the present invention will be describedwith reference to FIG. 1. The first embodiment hereinafter described isa motion image imaging apparatus containing the configuration of theimage processing apparatus 100 shown in FIG. 1, while the secondembodiment is a playback and receiving apparatus containing theconfiguration of the image processing apparatus 100 shown in FIG. 1.

The image processing apparatus 100 in accordance with the basicconfiguration of an embodiment of the present invention uses imageprocessing to generate images with reduced jerkiness. When displayingimaged motion images using a display apparatus, the motion images mayappear jerky and unnatural to the human eye. Consequently, in anembodiment of the present invention, jerkiness is reduced byappropriately adding motion blur using imaging information expressingparameters whereby the input motion images were imaged. In so doing, ahigh-quality image signal with few defects is generated and output.Thus, processing to improve image quality is realized.

The image processing apparatus 100 shown in FIG. 1 is provided with animage acquirer 1, a motion region determining unit 2, and a motion bluradder 3. The image acquirer 1 is configured to acquire and import imagedata into the image processing apparatus 100. For example, the imageacquirer 1 may acquire image data DD expressing motion images from animaging apparatus or similar device. The image acquirer 1 subsequentlysupplies the image data DD to the motion region determining unit 2 andthe motion blur adder 3.

There also exists, in association with the acquired image data DD,information expressing parameters whereby the image data DD was imaged.Such information exists in the form of imaging information CI, andincludes shutter speed, aperture value, focal length, and gyroscopeinformation, for example. The image acquirer 1 extracts the variousimaging information CI with respect to the acquired image data DD, andthen supplies the extracted information to the motion region determiningunit 2.

The motion region determining unit 2 detects moving objects on the basisof differences among adjacent frames in the image data DD. For example,given a particular region within a frame to be processed (i.e. a pixelunit or pixel block unit made up of a plurality of pixels), the motionregion determining unit 2 may determine whether that region is a motionregion or a still region. The motion region determining unit 2 makessuch a still/motion determination on a per-region basis. In addition, onthe basis of the imaging information CI, the motion region determiningunit 2 generates score values expressing the degree of jerkiness formoving objects, and then supplies the score values to the motion bluradder 3.

The motion blur adder 3 generates pseudo motion blur with respect toinput image data DD, on the basis of the score values provided by themotion region determining unit 2. As a result, output image data OD withreduced jerkiness is output. Although a variety of algorithms forgenerating motion blur may be considered, one particular example will begiven hereinafter as part of the description of the first embodiment.

In an image processing apparatus 100 as described above, the motion bluradder 3 adds motion blur to unit images in the input image data DD(image data for single frames, for example). Motion blur is added on thebasis of the results of the per-region still/motion determinationconducted by the motion region determining unit 2, as well as on thebasis of the imaging information CI. More specifically, on the basis ofboth the still/motion determination and the imaging information CI, themotion blur adder 3 adds motion blur to regions in the motion imageswhere jerkiness is likely to occur. In so doing, the addition ofsuitable blur is realized and jerkiness is reduced, without the heavyprocessing load incurred by detecting motion vectors for each regionwithin a frame.

[2. First Embodiment: Motion Image Imaging Apparatus]

[2-1: General Configuration of Motion Image Imaging Apparatus]

The configuration of a motion image imaging apparatus 200 in accordancewith the first embodiment will now be described with reference to FIG.2. The motion image imaging apparatus 200 herein images an object,performs processing to add motion blur to the imaged motion images asdescribed above, and then outputs the result.

More specifically, the motion image imaging apparatus 200 acquiresmotion images by imaging an object, and subsequently records the imagedimages to a DVD or similar recording medium 90, or transmits the imagedimages to an external device. As shown in FIG. 2, the motion imageimaging apparatus 200 is provided with imaging optics 10, which includean aperture mechanism 11, an optical lens system 12, and imagingelements 13.

The aperture mechanism 11 adjusts the amount of incident object light.The optical lens system 12 causes object light (whose amount has beenadjusted by the aperture mechanism 11) to be focused onto thelight-accepting surface of the imaging elements 13.

The imaging elements 13 receive light focused by the optical lens system12 onto the light-accepting surface thereof, and convert the receivedlight into an image signal. The imaging elements 13 then conductpredetermined signal processing and output the resulting motion imagesin the form of image data DD. The image data DD output from the imagingelements 13 is then supplied to a motion blur adder 30 and a motionregion determining unit 40.

The image data DD is herein assumed to be made up of unit images inprogressive format at a frame rate of 60 fps, for example. However, itshould be appreciated that the image data DD is not limited to being inprogressive format, and may also be made up of unit images that form aseries of motion images in interlaced format.

The motion image imaging apparatus 200 is also provided with a gyrosensor 14, which detects motion of the motion image imaging apparatus200 itself. For example, the gyro sensor 14 may detect rotationinformation in the horizontal and vertical directions. This rotationinformation is used for features such as image stabilization, whileadditionally being used as one component of the imaging information CIfor adding motion blur in the present example.

A controller 20 controls the overall operation of the motion imageimaging apparatus 200. The controller 20 may be realized by means of amicrocontroller, for example. The controller 20 conducts variouscontrols, such as control of imaging operations, signal processingcontrol, control of recording onto the recording medium 90, and controlof transmission to external devices, for example.

The control of imaging operations by the controller 20 includes controlof focusing and zooming by driving lenses with respect to the imagingoptics 10 when imaging. In addition, the controller 20 also controls theopening and closing of the aperture mechanism 11, the shutter speed atthe imaging elements 13, and exposure adjustment by means of signal gaincontrol, for example. The controller 20 may also conduct imagestabilization control, on the basis of the detected motion of the motionimage imaging apparatus 200 itself by the gyro sensor 14, for example.

Although detailed description of the various control operations isomitted herein, one process conducted by the controller 20 that isrelated to the image processing of the present example is the outputtingof the imaging information CI. More specifically, the controller 20 isable to obtain information regarding the acquired image data DD from thecontrol operations performed during imaging, as described above. Theinformation may include shutter speed information, aperture valueinformation, focal length information, and gyroscope rotationinformation, for example. The controller 20 then supplies this variousinformation to the motion region determining unit 40 in the form ofimaging information CI.

As later described with reference to FIG. 5, the motion regiondetermining unit 40 takes a given region within a frame (i.e., a pixelunit or pixel block unit made up of a plurality of pixels), and on thebasis of differences among adjacent frames in the image data DD,determines whether that region is a motion region or a still region. Inaddition, on the basis of the imaging information CI, the motion regiondetermining unit 2 generates score values E expressing the degree ofjerkiness for moving objects, and then supplies the score values to themotion blur adder 30.

The configuration and processing of the motion blur adder 30 will belater described in detail and with reference to FIGS. 6 to 18. Briefly,the motion blur adder 30 generates pseudo motion blur with respect toinput image data DD, on the basis of the score values provided by themotion region determining unit 40. In this case, the motion blur adder30 adds motion blur to regions where jerkiness is likely to occur, inaccordance with the magnitude of the jerkiness. By thus adaptivelyadding motion blur, the motion blur adder 30 outputs output image dataOD wherein jerkiness has been reduced.

After the motion blur adder 30 adaptively adds motion blur, theresulting output image data OD is compressed by a motion image codingprocessor 50. The compressed data stream is then supplied to thetransmit processor 60 or the recording processor 70.

The transmit processor 60 encodes and modulates the compressed datastream for transmission, and then outputs the data stream bytransmitting to an external device.

The recording processor 70 encodes and modulates the compressed datastream for recording, and then records the data stream onto an opticaldisc or other recording medium 90, for example.

The controller 20 additionally provides the imaging information CI tothe transmit processor 60 and the recording processor 70. The transmitprocessor 60 and the recording processor 70 are thus able torespectively transmit or record the transmit data or recording data withthe imaging information CI superimposed thereon.

[2-2: Imaging Information and Score Value Generation]

In a motion image imaging apparatus 200 configured like the above, themotion region determining unit 40 determines motion regions in eachframe of the image data DD, and motion blur is subsequently added to themotion regions by the motion blur adder 30. However, the motion regiondetermining unit 40 also generates score values for adding motion blur,and then provides the score values to the motion blur adder 30. Thescore values are generated using the imaging information CI.

The imaging information CI provided to the motion region determiningunit 40 by the controller 20 is herein assumed to be shutter speedinformation S, aperture value information F, focal length information D,and gyroscope rotation information GX (horizontal rotation) and GY(vertical rotation).

Additionally, the score values E generated by the motion regiondetermining unit 40 are herein assumed to be information indicating howmuch motion blur is to be added to each motion region within a frame,for example.

FIGS. 3A, 3B, and FIGS. 4A to 4C illustrate concepts whereby the motionregion determining unit 40 generates score values on the basis ofimaging information CI. FIGS. 3A, 3B and FIGS. 4A to 4C are graphs usedby the motion region determining unit 40 to control score values E inaccordance with imaging information CI provided by the controller 20. Ineach of the graphs shown in FIGS. 3A, 3B, and FIGS. 4A to 4C, thevertical axis represents a particular score value E and its range ofhigh and low values. Herein, a high score value E is taken to mean thatthe motion of the object is large, jerkiness is significant, and thus alarge amount of motion blur is to be added.

In FIG. 3A, the horizontal axis represents the shutter speed S as onetype of information included in the imaging information CI. As describedearlier, the controller 20 is able to obtain the shutter speed used forimaging from electronic shutter function control information used whendriving the imaging elements 13.

When the shutter speed indicated by the supplied shutter speedinformation is faster than the frame rate of the image data DD, themotion region determining unit 40 causes score values in motion regionsto increase by a power of the speed. When the shutter speed indicated bythe shutter speed information is equal to the frame rate, the scorevalue lowers, and the processing to add motion blur is invalidated.

Herein, the frame rate is expressed in frames per second (fps). A framerate of 60 fps thus equates to 60 frames in 1 second. In this case, theexposure time of the electronic shutter for a single frame at theimaging elements normally has a maximum of 1/60 sec. This is the shutterspeed referred to herein. Typically, the electronic shutter control isconducted for functions such as exposure control. In the present case,the exposure period is varied among single frame periods. For example,exposure adjustment may be conducted such that the exposure time (i.e.,shutter speed) within 1/60 sec is varied among 1/120 sec, 1/180 sec, and1/1000 sec, for example. The motion region determining unit 40 thusgenerates score values E by comparing the above shutter speed S to theoriginal shutter speed expressing the frame rate (such as 1/60 sec).

Taking S to be the shutter speed, and Es to be a score value dependenton the shutter speed S, then

Es=fs(S)

can be used to manipulate score values. If the function fs(x) is takento be a linearly increasing function, then a plot like that shown by thesolid line in FIG. 3A is realized. However, a non-linear function mayalso be used, and score values E dependent on the shutter speed S mayalso be generated like that shown by the broken lines in FIG. 3A, forexample.

FIG. 3B is a graph used by the motion region determining unit 40 tocontrol score values in accordance with an aperture value F provided bythe controller 20. The controller 20 is able to obtain the aperturevalue F used for imaging from the control information of the imagingoptics 10.

When the aperture value F is small, the motion region determining unit40 determines that the depth of field is shallow, the area around theobject is easily blurred, and thus jerkiness is unlikely to occur. Forthis reason, the motion region determining unit 40 lowers the scorevalue E. In contrast, when the aperture value F is large, the motionregion determining unit 40 determines that the depth of field is deep,the image is sharp throughout, and thus jerkiness is likely to occur.For this reason, the motion region determining unit 40 raises the scorevalue E.

If Ef is herein taken to be the score value dependent on the aperturevalue F, then

Ef=ff(F)

can be used to manipulate score values. It should be appreciated thatthe function ff(x) is not limited to being a linearly increasingfunction as shown in FIG. 3B, and may also be a non-linear function.

FIG. 4A is a graph used by the motion region determining unit 40 tocontrol score values in accordance with a focal length D provided by thecontroller 20. The controller 20 is able to obtain the focal length Dused for imaging from the control information of the imaging optics 10.

When the focal length D is short (i.e., wide-angle), then the depth offield deepens, and jerkiness is likely to occur when considering thisfactor alone. However, in this case, camera unsteadiness and objectmotion is relatively small on screen, and thus the motion regiondetermining unit 40 determines that jerkiness is unlikely to occuroverall, and lowers the score value.

In contrast, when the focal length D is long (i.e., telescopic), thenthe depth of field narrows, and jerkiness is unlikely to occur whenconsidering this factor alone. However, in this case, cameraunsteadiness and object motion is relatively large on screen, and thusthe motion region determining unit 40 determines that jerkiness islikely to occur overall, and raises the score value.

If Ed is herein taken to be the score value for the focal length D, then

Ed=fd(D)

can be used to manipulate score values. It should be appreciated thatfd(x) is not limited to being a linearly increasing function as shown inFIG. 4A, and a non-linear function may also be used.

FIG. 4B is a graph used by the motion region determining unit 40 tocontrol a horizontal score value EX in accordance with gyroscopeinformation (specifically, the horizontal rotation GX) provided by thecontroller 20. The controller 20 is able to acquire the horizontalmotion of the motion image imaging apparatus 200 itself from theinformation detected by the gyro sensor 14.

When the horizontal rotation GX (i.e., rotation about the vertical axis)is small, the motion region determining unit 40 determines thaton-screen motion is small and jerkiness is unlikely to occur, and thuslowers the horizontal score value EX. In contrast, when the rotation GXis large, the motion region determining unit 40 determines thatjerkiness is likely to occur, and thus raises the horizontal score valueEX. If Egx is herein taken to be the horizontal score value for therotation GX, then

Egx=fgx(GX)

can be used to manipulate score values. It should be appreciated thatthe function fgx(x) is not limited to being a linearly increasingfunction as shown in FIG. 4B, and may also be a non-linear function.

FIG. 4C is a graph used by the motion region determining unit 40 tocontrol a vertical score value EY in accordance with gyroscopeinformation (specifically, the vertical rotation GY) provided by thecontroller 20. The controller 20 is similarly able to acquire thevertical motion of the motion image imaging apparatus 200 itself fromthe information detected by the gyro sensor 14.

When the vertical rotation GY (i.e., rotation about the horizontal axis)is small, the motion region determining unit 40 determines thaton-screen motion is small and jerkiness is unlikely to occur, and thuslowers the vertical score value EY. In contrast, when the rotation GY islarge, the motion region determining unit 40 determines that jerkinessis likely to occur, and thus raises the vertical score value EY. If Egyis herein taken to be the horizontal score value for the rotation GY,then

Egy=fgy(GY)

can be used to manipulate score values. It should be appreciated thatthe function fgy(y) is not limited to being a linearly increasingfunction as shown in FIG. 4B, and may also be a non-linear function.

The motion region determining unit 40 thus generates score values E inaccordance with the imaging information CI using methods like thosedescribed by way of example above.

FIG. 5 illustrates an exemplary configuration of the motion regiondetermining unit 40. The motion region determining unit 40 is providedwith a motion region detector 41, frame memory 42, and a score valuegenerator 43.

The motion region detector 41 detects regions with movement (i.e. motionregions) by solving for the difference between the current frame imageand the immediately previous frame image in the input image data DD.Herein, a region may be a pixel unit, or a pixel block unit made up of aplurality of pixels.

More specifically, while storing an image of the current frame in theframe memory 42, the motion region detector 41 reads the image of theprevious frame from the frame memory 42, and then conducts differentialcalculation on a per-region basis. The calculation results aresubsequently used to output detection result values indicated whethereach respective region is a motion region or a still region. Forexample, the motion region detector 41 may generate detection resultinformation for each region in the current frame, wherein motion regionsare represented as 1, and still regions are represented as 0. Thisdetection result information is then supplied to the score valuegenerator 43.

The score value generator 43 then generates score values with respect tothe motion regions (i.e., the regions having a value of 1 in thedetection results) specified by the detection results output from themotion region detector 41. The score value generator 43 generates scorevalues on the basis of the imaging information CI (i.e., the shutterspeed S, the aperture value F, the focal length D, and the rotations GXand GY) provided by the controller 20.

At this point, the control with respect to each subset of imaginginformation CI follows the graphs shown in FIGS. 3A to 4C. The finalscore values are output by combining operations using the above graphs.

As one example, final score values Ex and Ey may be solved for using

Ex=ws×Es+wf×Ef+wd×Ed+wgx×Egx

Ey=ws×Es+wf×Ef+wd×Ed+wgx×Egy

wherein the score value Ex is a value indicating the amount of motionblur to be added in the horizontal direction, and the score value Ey isa value indicating the amount of motion blur to be added in the verticaldirection. In the above formulas, ws is a weighting coefficient appliedto the score value Es based on the shutter speed S, wf is a weightingcoefficient applied to the score value Ef based on the aperture value F,wd is a weighting coefficient applied to the score value Ed based on thefocal length D, wgx is a weighting coefficient applied to the scorevalue Egx based on the horizontal rotation GX, and wgy is a weightingcoefficient applied to the score value Egy based on the verticalrotation GY.

By using weighted addition like that shown by way of example above, thescore value generator 43 generates a score value Ex for the horizontaldirection and a score value Ey for the vertical direction, and thenoutputs these values to the motion blur adder 30. On the basis of thesupplied score values E (Ex and Ey in this case), the motion blur adder30 adds motion blur as described hereinafter.

In this way, the motion region determining unit 40 first detects motionregions within a frame, and then generates score values for each motionregion, with the values being adjusted on the basis of the imaginginformation CI.

For example, for still regions within a frame, the detection result (0)from the motion region detector 41 may be used for the score value Eas-is (i.e., E=0), thereby enabling the motion blur adder 30 torecognize still regions as regions to which motion blur is not added.

For motion regions, the score value generator 43 may used the detectionresult (1) from the motion region detector 41 to perform computations onthe basis of the imaging information CI, such as that given by the aboveformulas for Ex and Ey, and thereby adjust the score values inaccordance with the imaging information CI.

In other words, for motion regions, the above values Ex and Ey solvedfor on the basis of the imaging information CI are taken to be the scorevalues, and then passed to the motion blur adder 30 as informationindicating the amount of motion blur to be added.

It should be appreciated that the calculations used to solve for thescore values Ex and Ey as shown above are examples, and that a varietyof other calculations are also conceivable. For example, weightedmultiplication may be used.

Furthermore, although both horizontal and vertical score values Ex andEy are solved for herein by using rotations detected by the gyro sensor14 as horizontal and vertical rotations, a single score value E may alsobe computed without applying directionally-based distinctions.

In contrast, it is also conceivable to detect motion directions such as45° and 135°, and then compute respective score values for thehorizontal, vertical, 45°, and 135° directions.

Moreover, a score value may be computed without using all of the shutterspeed information S, aperture value information F, focal lengthinformation D, and gyroscope rotation information GX (horizontalrotation) and GY (vertical rotation). In other words, a score value Emay be computed using just a portion of the above.

In addition, rather than treating the detection results from the motionregion detector as binary values (i.e., 0 or 1), the direction resultsmay be treated as many-valued. In this case, the detection results maybe 0 for still regions, with the value becoming larger for regions witha large amount of motion, and smaller for regions with a small amount ofmotion. Taking the detection results as M, the final score values maythen be computed by multiplying the score values Ex and Ey solved for asabove by the value of M.

Moreover, in the above example, the respective score values Es, Ef, Ed,Egx, and Egy are derived from the shutter speed information S, aperturevalue information F, focal length information D, and gyroscope rotationinformation GX and GY by means of functions, with the score values Es,Ef, Ed, Egx, and Egy being subsequently combined by weighted addition.However, it is also possible to compute a score value E by directlyusing the shutter speed information S, aperture value information F,focal length information D, and gyroscope rotation information GX andGY.

[2-3: Exemplary Configuration and Operation of Motion Blur Adder]

The motion blur adder 30 takes image data DD supplied by the imagingelements 13, and then adds motion blur on the basis of the score valuesE from the motion region determining unit 40. The exemplaryconfiguration and operation of the motion blur adder 30 will now bedescribed.

FIG. 6 illustrates an exemplary configuration of the motion blur adder30. The motion blur adder 30 accepts the image data DD and the scorevalues E from the motion region determining unit 40 as input, addsmotion blur to the image data DD for each frame constituting the inputmotion images, and then outputs output image data OD where jerkiness hasbeen reduced.

As shown in FIG. 6, the motion blur adder 30 is provided with a reducedimage generator 32. The reduced image generator 32 adds motion blur topixels that have been selected from the processed frame by filtering orsimilar processing, and then generates one or more reduced layers madeup of the pixels obtained as a result.

In addition, the motion blur adder 30 is also provided with a motionblur rendering processor 31. The motion blur rendering processor 31computes and outputs pixel values for each pixel constituting the outputimage. The motion blur rendering processor 31 computes pixel values onthe basis of the score values E from corresponding pixels in theprocessed frame as well as corresponding pixels in the reduced layersgenerated by the reduced image generator 32.

Before describing the specific processing executed by each of the abovecomponents, the processing to add motion blur that is conducted by themotion blur adder 30 will be summarized.

The motion blur adder 30 adds a suitable amount of motion blur to eachpixel using the reduced image generator 32 and the motion blur renderingprocessor 31, and then outputs images with reduced jerkiness. FIG. 7summarizes the processing to add motion blur. More specifically, FIG. 7summarizes the processing executed in the reduced image generator 32 andthe motion blur rendering processor 31 shown in FIG. 6.

The original image (base layer) shown in FIG. 7 refers to the image datafor a single processed frame, in the form of input image data DD. Thereduced image generator 32 first executes spatial filtering processing(to be later described in detail) with respect to the original imageexpressing the processed frame, and creates a reduced layer 1 as aresult. The reduced layer 1 is an image that has been reduced from theoriginal image by a fixed ratio, wherein the pixel count has decreased.

Subsequently, the reduced image generator 32 executes similar spatialfiltering processing with respect to the obtained reduced layer 1, andcreates a reduced layer 2 as a result. Using the above procedure, thereduced image generator 32 uses recursive filter processing to generatea predetermined number of layers of reduced images (i.e., a number ofreduced layers 1 to n).

Using the hierarchical stack of reduced images (i.e., the reduced layers1 to n) generated by the above procedure, the motion blur renderingprocessor 31 generates and outputs images more natural to human eye,wherein motion blur has been adaptively added and jerkiness has beenreduced.

For example, given the respective pixels in the original image shown inFIG. 7, the motion blur rendering processor 31 extracts pixel valuesfrom the base layer (i.e., the original image), the reduced layer 1, andthe reduced layer 2, wherein the extracted pixel values are those ofpixels positioned at locations corresponding to that of a particularpixel to which motion blur is to be added. The motion blur renderingprocessor 31 then executes weighted addition of the extracted pixelvalues according to a given rendering method.

Since the number of pixels differs for each layer, a pixel positioned ata location corresponding to that of the particular pixel targeted formotion blur might not exist in each reduced layer. In this case,interpolation processing (to be hereinafter described in detail) is usedto interpolate and extract a pixel value from surrounding pixels.

The weighting coefficients (w0, w1, and w2) corresponding to therespective pixel values extracted from the base layer and each reducedlayer are dependent on, and determined by, the score value E.

For example, when the score value E is large (i.e., when the amount ofmotion blur to be added is large), the weighting coefficient isincreased for the pixel extracted from the reduced layer in the upperlayer. In the example shown in FIG. 7, the above corresponds toincreasing the weighting coefficient for the pixel extracted fromreduced layer 2.

In contrast, when the score value E is small (i.e., when the amount ofmotion blur to be added is small), the weighting coefficient isincreased for the pixel extracted from the reduced layer in the lowerlayer. In the example shown in FIG. 7, the above corresponds toincreasing the weighting coefficient for the pixel extracted from thebase layer.

As a result of such weighted addition, an image with reduced jerkinessis generated.

The specific processing conducted by the reduced image generator 32 willnow be described. The processing conducted by the reduced imagegenerator 32 involves executing spatial filtering with respect to inputimage data DD. Furthermore, similar spatial filtering is repeated withrespect to the reduced image obtained as a result (even if the result ofthe first spatial filtering is not structured as an image, the pluralityof pixels obtained as a result thereof is used). In so doing, a layeredstack of reduced image data RSD is generated.

The above spatial filtering is conducted using pixel information for aregion containing a plurality of pixels in the pre-filtering image. Theimage is then reduced by a fixed ratio (comparing images before andafter processing), and an image with a reduced pixel count is generated.For this reason, each filtered pixel is output as an approximation of acorresponding region in the pre-filtering image.

Hereinafter, the above spatial filtering will be described in detail,using a very basic filtering method by way of example. FIG. 8illustrates the relationship between pixels before and after reductionin a specific example of filter processing used to generate reducedimages.

In the example shown in FIG. 8, filtering is executed with respect tothe pre-reduction image using a 4×4 pixel block herein labeled as thedivision region R11, and wherein a pixel P11 in the reduced image isoutput as a result. The pixels P12 and P21 are shown to exist in asimilar relationship with the division regions R12 and R21. As a result,the pixel count of the reduced image becomes ¼ of that of thepre-reduction image in both the vertical and horizontal directions.

It should be appreciated that the use of 4×4 pixel blocks is merely oneexample. If similar processing is conducted using N×N pixels blocks,then a reduced image can be generated having a pixel count of 1/N inboth the vertical and horizontal directions.

Assuming, by way of example, that a box filter or similar low passfilter is conducted with respect to the division region R11, the pixelvalues of nearby pixels are averaged, and a blurred pixel P1 isgenerated as a result.

If the reduced layer 1 shown in FIG. 7 is taken to be a reduced imageobtained as above, then spatial filtering similar to that shown in FIG.8 is executed with respect to the reduced layer 1, thereby generatingthe reduced layer 2.

In this case, by filtering the reduced layer 1 using 4×4 pixel divisionregions, each pixel in the reduced layer 2 is generated as a blurredpixel representing the average of the pixel values within a 16×16 pixelregion in the original image.

After generating the reduced layer 2, it is also possible to generate areduced layer 3 and subsequent layers as appropriate, using the sameprocedure. The process itself does not change, even if the number oflayers is increased.

In this way, by executing spatial low-pass filtering with respect to aninput image in the reduced image generator 32, a resolution image isgenerated as a result, and additional images even lower in resolutionare generated by executing similar processing with respect to agenerated low-resolution image. A layered stack of reduced image dataRSD is thus generated by executing multiple-resolution filtering likethe above.

As another example, a filter method for generatingdirectionally-selected reduced images will now be described withreference to FIGS. 9 and 10. FIG. 8 illustrates division regions in theform of 4×4 pixel, square regions by way of example. However, thedivision regions whereby filtering is executed are not limited to beingsquare.

On the contrary, problems related to image quality might occur if motionblur is added in the subsequent motion blur rendering process with asquare or similar filter using 4×4 pixels as shown in FIG. 8. Suchproblems occur due to a phenomenon wherein, instead of just the desiredmotion blur, motion blur in the direction orthogonal to the motion andgenerated as a result of lowering the resolution becomes dominant.

For example, consider the case wherein it is desirable to add motionblur corresponding to motion in the horizontal direction. However, thereduced images that are used have been subjected to low-pass filteringin not only the horizontal direction (i.e., the direction of motionblur), but also the vertical direction. For this reason, the verticalresolution of the resulting image is decreased.

In other words, when reducing jerkiness by adding motion blurcorresponding to the direction of motion shown in an image, blur isadded in both the motion direction and the orthogonal direction, whichmay result in a visually unsuitable image.

In order to avoid such an event, filtering to generate reduced layers isconducted so as to take into account the direction of the motion blur tobe finally added. With this method, reduced layers are preparedindependently for each direction in which motion blur is to be added.

FIG. 9 illustrates the relationship between pixels before and afterreduction in a specific example of filter processing for generatingreduced images used to add motion blur in the horizontal direction. InFIG. 9, a 4×1 tap rectangular filter is applied to input image data DD(i.e., the base layer), thereby generating a horizontally reduced ¼image whose pixel count is ¼ of the base layer in the horizontaldirection, and unchanged in the vertical direction.

For example, filtering the regions R₀ 11, R₀ 12, and R₀ 21 in the baselayer results in the pixels P₁ 11, P₁ 12, and P₁ 21 in the horizontallyreduced ¼ image. It should be appreciated that use of 4×1 pixel regionsis merely one example, and may be changed to suit the desired reductionratio. The filter itself may be a box filter or similar low-pass filter,for example.

Similar filter processing is then executed with respect to thehorizontally reduced ¼ image generated above, thereby generating ahorizontally reduced 1/16 image whose pixel count is 1/16 of the baselayer in the horizontal direction, and unchanged in the verticaldirection.

For example, filtering the region R₁ 11 in the horizontally reduced ¼image results in the pixel P₂ 11 in the horizontally reduced 1/16 image.In this way, a multi-layered stack of images reduced in the horizontaldirection only is generated. The images can then be used to add motionblur in the horizontal direction, while also preventing resolution lossin the vertical direction.

FIG. 10 illustrates the relationship between pixels before and afterreduction in a specific example of filter processing for generatingreduced images used to add motion blur in the vertical direction.Similarly to FIG. 9, FIG. 10 shows a 1×4 tap rectangular filter beingfirst applied to input image data DD (i.e., the base layer), therebygenerating a vertically reduced ¼ image whose pixel count is ¼ of thebase layer in the vertical direction, and unchanged in the horizontaldirection. The filter itself may be a box filter or similar low-passfilter, for example. For example, filtering the regions R₀ 11, R₀ 12,and R₀ 21 in the base layer results in the pixels P₁ 11, P₁ 12, and P₁21 in the vertically reduced ¼ image.

Similar filter processing is then executed with respect to thevertically reduced ¼ image generated above, thereby generating avertically reduced 1/16 image whose pixel count is 1/16 of the baselayer in the vertical direction, and unchanged in the horizontaldirection. For example, filtering the region R₁ 11 in the verticallyreduced ¼ image results in the pixel P₂ 11 in the vertically reduced1/16 image. In this way, a multi-layered stack of images reduced in thevertical direction only is generated. The images can then be used to addmotion blur in the vertical direction, while also preventing resolutionloss in the horizontal direction.

The above describes the vertical and horizontal directions by way ofexample, but it should be appreciated that filters for generatingreduced images in other directions may also be adopted, such as filtersfor the diagonals 45° and 135°, or other arbitrary angles.

In the method described thus far, reduced images are generatedindependently for adding motion blur in respective directions. For thisreason, in the subsequent motion blur rendering process, pixels areadaptively selected or composited from reduced images in respectivedirections, in accordance with the direction of the motion blur to beadded to each particular pixel. A method for the above will be given inthe description of the motion blur rendering processor 31.

The foregoing thus describes filtering for generating reduced layerswhile taking into account the direction of the motion blur to be added.In order to describe the above process in further detail, a specificexemplary configuration will now be given, and description hereinafterwill successively make use thereof. The specific example is describedbelow.

Specific Example: Parameters 1

Directions of reduced image generation: horizontal, vertical

No. of reduced image layers: three horizontal and three vertical (i.e.,up to the reduced layer 3)

Direction of generated motion blur: the greater of the absolute valuesof the horizontal and vertical translational speeds at the time ofmotion blur rendering (to be later described in detail)

Post-filter reduction ratios (with respect to pre-filter image):

Reduced layer 1: ¼

Reduced layer 2: 1/16

Reduced layer 3: 1/64

Given the above specifications, the maximum amount of motion blur thatcan be added is equivalent to a translation of ±64 pixels in thehorizontal and vertical directions, respectively.

FIG. 11 illustrates an exemplary configuration of the reduced imagegenerator 32 given settings indicated by the above parameters 1. FIG. 11illustrates a process flow in the case where processing to generatedreduced layers is executed using one-dimensional filters in thehorizontal and vertical directions, respectively. The reduced layersrespectively generated are then output as the reduced image data RSD.

More specifically, input image data DD is filtered by a 1D filter 21-1to generate a horizontally reduced ¼ image. The result from the 1Dfilter 21-1 is additionally filtered by a 1D filter 21-2 to generate ahorizontally reduced 1/16 image. The result from the 1D filter 21-2 isadditionally filtered by a 1D filter 21-3 to generate a horizontallyreduced 1/64 image.

Meanwhile, the input image data DD is also filtered by a 1D filter 22-1to generate a vertically reduced ¼ image. The result from the 1D filter22-1 is additionally filtered by a 1D filter 21-2 to generate avertically reduced 1/16 image. The result from the 1D filter 221-2 isadditionally filtered by a 1D filter 22-3 to generate a verticallyreduced 1/64 image. The six sets of reduced image data RSD obtained as aresult of the above are then output.

The processing conducted by the 1D filters (21-1 to 22-3) shown in FIG.11 is herein taken to be recursive filter processing realized bysoftware-based computations and illustrated as a process flow expressedusing blocks. However, the above configuration may also be taken to be ahardware configuration. The following FIG. 12 is similar.

FIG. 12 is an extended generalization of the process flow shown in FIG.11. Given the settings indicated by the above parameters 1, the numberof layers to be generated in the horizontal and vertical directions isfixed at three. However, FIG. 12 shows a generalized process flow notlimited to the above parameter, wherein the number of horizontallyreduced layer layers is expressed as Mx, and the number of verticallyreduced layer layers is expressed as My.

By means of the 1D filters 21-1 to 21-Mx, Mx layers of horizontallyreduced images are generated. Likewise, by means of the ID filters 22-1to 22-My, My layers of vertically reduced images are generated. Theresulting images are then output as the reduced image data RSD.

Furthermore, although omitted from FIG. 12 and the description herein,it is also possible to extend the configuration such that the directionsof reduced image generation include not only horizontal and vertical,but also diagonal directions.

The foregoing thus describes a processing sequence related to a methodfor independently generating reduced images used to add motion blur inrespective directions. Hereinafter, an actual filter used whengenerating reduced images for respective directions will be described.

In the description thus far, the use of a 4-tap box filter is given byway of example. This is because a 4-tap box filter is one of the mosteasily realized methods when using a 4-pixel block filter correspondingto a reduction ratio of ¼. When the underlying principle behind motionblur is taken into account, a box filter, being equivalent to a movingaverage filter, should closely resemble actual motion blur in principle.However, it is also possible to modify the filter in order to improvethe image quality that is ultimately obtained.

FIG. 13 illustrates specific examples of filter shapes used for reducedimage generation processing. Given the parameters 1 described earlier,the filter is assumed to reduce an image to ¼ size in either thehorizontal or vertical direction. The horizontal axis indicatescoordinates for the case wherein sets of 4 pixels are considered to besingle blocks, while the vertical axis indicates the weight by whichindividual pixels are multiplied in the filter.

In addition to a 4-tap box filter, FIG. 13 also illustrates a 7-tap tentfilter, a 10-tap rank filter, and a 13-tap Gaussian filter by way ofexample, and also shows that it is possible to adopt an FIR low-passfilter if available.

When the reduction ratio by the reducing filter is 1/N, the filter tapnumber is N-tap or greater. (For example, a 4-tap filter is used for areduction ratio of ¼, wherein 1 pixel is output with respect to 4 inputpixels.) By further increasing the number of taps, improved imagequality can be expected. This is because an increased number of tapsalleviates image quality loss due to the folding phenomenon, which ismore likely to occur with lower tap numbers. Furthermore, in addition toincreasing the filter tap number, pixels belonging to adjacent divisionregions become overlapped and filtered, but this does not pose aproblem.

FIG. 14 is a diagram for explaining the positional relationship ofpixels in respective reduced images used when filtering. The way inwhich pixels from each reduced image are subjected to weighted additionin order to compute the pixel value of an output pixel and generate ahorizontally reduced image will now be described with reference to FIG.14. A filter tap number of 7 is used herein by way of example. Sincevertically reduced images can also be generated according to sameprocedure, further description of the vertical case is omitted.

In FIG. 14, the pixels in each reduced image are numbered. Hereinafter,these numbers will be used to designate particular pixels in reducedimages of particular layers. For example, the 79th pixel in the 1/16reduced image is represented as L2(79). Pixels in the input image dataDD (i.e., the base layer) are expressed as L0(n), where n is a pixelnumber.

The pixel numbers in each reduced image correspond to positions on thebase layer, and are created by executing filtering whose phase iscentered about the shaded pixels in each reduced layer. For example, thepixel L1(7) in the first, ¼ reduced layer is generated as a result ofapplying a 7-tap filter to the pixels L0(4) to L0(10) on the base layer.

In this way, the filter is shifted by 4 pixels in the respective reducedimages during execution. In so doing, no problem is posed even if thepixels used overlap with those of the adjacent filter, and the reducedlayer for the next layer (being reduced by ¼ as a result of applying thefilter) is created. Obviously, it is also possible to realize differentreduction ratios with a similar method.

The specific processing conducted by the motion blur rendering processor31 will now be described. The motion blur rendering processor 31 takesthe input image data DD as well as the layered stack of reduced imagesgenerated by the reduced image generator 32, and uses the images tocompute the value of each pixel in an output image that contains motionblur.

FIG. 15 is a schematic diagram illustrating a process flow for motionblur rendering. Image data DD, reduced image data RSD, and score valuesE (such as a horizontal score value Ex and a vertical score value Ey)are input into the motion blur rendering processor 31, and output imagedata OD wherein a suitable amount of motion blur has been added to eachpixel is then output.

First, rendering parameters are extracted from the score values Ex andEy (S1). The rendering parameters specify the motion blur to add toparticular pixels (i.e., the direction of the motion blur, and thenumber of pixels across which the motion blur translates).

In other words, the rendering parameters are data specifying the amountof weight by which to multiply individual pixels in the image data DDand the reduced image data RSD in the weighted addition of thesubsequent rendering process. The rendering parameters will be laterdescribed more fully by means of a specific example.

Pixel compositing is then conducted with respect to the reduced imagedata RSD, wherein pixel values are generated by interpolation of pixelsin each reduced image that positionally correspond to a particular pixelin the image data DD (S2). The reasoning behind the above processing anda specific example thereof will be later given in detail.

Finally, rendering is conducted, wherein weights are respectivelyapplied to a particular pixel in the image data DD as well as toindividual composite pixels generated from the reduced image data RSD.The weighted pixels are then added together to compute the pixel valueof the output pixel corresponding to the particular pixel in the imagedata DD. Weighting information is one of the rendering parameters inputearlier.

FIG. 16 is a diagram of the configuration of the motion blur adder 30,illustrating in detail the processing blocks of the motion blurrendering processor 31 when given the parameters 1 specified earlier. Asshown in FIG. 16, the reduced image data RSD created by the reducedimage generator 32 contains three layers of reduced images for thehorizontal and vertical directions, respectively.

In the motion blur rendering processor 31, when a particular pixel to beprocessed from the image data DD is input, pixels from the three layersof horizontally reduced images are respectively input into pixelcompositors 33-1 to 33-3 at the same time, and processing to computecomposite pixel values (to be hereinafter described) is conducted.

In addition, when a particular pixel to be processed from the image dataDD is input, pixels from the three layers of vertically reduced imagesare respectively input into pixel compositors 32-1 to 32-3 at the sametime, and processing to compute composite pixel values (to behereinafter described) is conducted.

The composite pixels corresponding to each reduced image and computed bythe pixel compositors 33-1 to 33-3 and 32-1 to 32-3 are then input intoa selector 34.

Meanwhile, the score values E (in this case, Ex and Ey) are input into ascore value determining unit 35. The parameters 1 specified earlier areparameters whereby motion blur is selectively added to each pixel in oneof either the horizontal direction or the vertical direction.Consequently, on the basis of the score values Ex and Ey, the scorevalue determining unit 35 generates selection information specifyingwhether motion blur is to be added in the horizontal direction or thevertical direction, as well as weighting information specifying theweights by which to multiply each composite pixel value for givendirections of motion blur. The selection information is subsequentlysupplied to the selector 34, while the weighting information is suppliedto a pixel weighting processor 36.

On the basis of the selection information input from the score valuedetermining unit 35, the selector 34 selects pixel values from among thecomposite pixel values for only one of either the horizontal directionor the vertical direction, and then outputs the selected pixel values tothe pixel weighting processor 36. In other words, when given theparameters 1, six pixel values are input into the selector 34, and threepixel values are output.

The particular pixel to be processed from the input image data DD thatwas previously input is then input into the pixel weighting processor36. Additionally, the respective pixels values that were directionallyselected from among the composite pixel values are also input, asdescribed above. Weighting information corresponding to each pixel valueis also input from the score value determining unit 35. On the basis ofthe weighting information, the pixel weighting processor 36 executesweighted pixel addition to be later described in detail, and therebycomputes an output pixel value.

The processing conducted by the pixel compositors 33-1 to 33-3 and 32-1to 32-3 will now be described with reference to FIG. 17. FIG. 17 is adiagram for explaining a specific example of pixel compositing andweighted pixel addition. Individual pixels in the image data DD and thereduced image data RSD are illustrated similarly to those shown in FIG.14.

Although the computation of composite pixel values conducted by thepixel compositors 33-1 to 33-3 and 32-1 to 32-3 is performed withrespect to pixels input from horizontally and vertically reduced images,respectively, FIG. 17 just illustrates the case wherein motion blur isapplied in the horizontal direction (i.e., just the pixel compositors33-1 to 33-3). However, the description hereinafter may be similarlyapplied to the vertical case (i.e., the pixel compositors 32-1 to 32-3).

First, the reasoning behind the computation of composite pixel valueswill be given. As described earlier, the subsequent pixel weightingprocessor 36 respectively applies weights to both a particular pixel inthe image data DD as well as respective composite pixels generated fromthe reduced image data RSD. The weighted pixels are then added togetherto compute the pixel value of an output pixel that corresponds to theparticular pixel in the image data DD. In some cases, however, pixelscorresponding to the particular pixel in the image data DD do not existin each reduced image. For this reason, it is preferable to compute acorresponding pixel in each reduced image by interpolation.

In FIG. 17, for example, there do not exist pixels in the respectivereduced images that positionally correspond to the pixel L0(20) in theimage data DD. This is because when the image data DD is filtered andthe first, ¼ reduced image is generated, the pixel L0(20) is not a phasecenter of the filter. As a result, a positionally corresponding pixeldoes not exist in the ¼ reduced layer and subsequent layers.

For this reason, some type of interpolation is used to estimate pixelvalues L1(20)′, L2(20)′, and L3(20)′ in the reduced images thatcorrespond to the particular pixel in the image data DD. Herein, thecase wherein a linear interpolation method is used to compute L1(20)′,L2(20)′, and L3(20)′ is considered by way of example.

At this point, the distances between a particular pixel and thecoordinates of a pixel in the input image data DD that is used as aphase center when generating respective reduced images can be expressedas d₁(20), d₂(20), and d₃(20), as shown in FIG. 17. The above distancesare measured to the closest phase center pixel existing to the left ofthe current pixel. In the present case, the composite pixel valuesL₁(20)′, L₂(20)′, and L₃(20)′ existing in respective reduced images andcorresponding to the pixel L₀(20) in the input image data DD can beexpressed as:

(Equation 1)

Example of composite pixel in the ¼ reduced image

${L_{1}(20)}^{\prime} = {{\frac{4 - {d_{1}(20)}}{4}*{L_{1}(19)}} + {\frac{d_{1}(20)}{4}*{L_{1}(23)}}}$

Example of composite pixel in the 1/16 reduced image

${L_{2}(20)}^{\prime} = {{\frac{16 - {d_{2}(20)}}{16}*{L_{2}(15)}} + {\frac{d_{2}(20)}{16}*{L_{2}(31)}}}$

Example of composite pixel in the 1/64 reduced image

${L_{3}(20)}^{\prime} = {{\frac{64 - {d_{3}(20)}}{64}*{L_{3}\left( {- 1} \right)}} + {\frac{d_{3}(20)}{64}*{L_{3}(63)}}}$

In general, the pixel value L_(K)(N)′ of a composite pixel existing inthe Kth reduced image (a 1/R reduced layer) and corresponding to the Nthpixel L₀(N) in the input image data DD can be expressed as:

$\begin{matrix}{{L_{K}(N)}^{\prime} = {{\frac{R - {d_{k}(N)}}{R}*{L_{K}\left( {\langle N\rangle}_{K} \right)}} + {\frac{d_{K}(N)}{R}*{L_{K}\left( {{\langle N\rangle}_{K} > {+ 1}} \right)}}}} & \left( {{Equation}\mspace{14mu} 2} \right)\end{matrix}$

wherein <N>_(K) is the phase center pixel closest to N in the Kthreduced image and existing to the left of the corresponding Nth pixel inthe input image data DD.

As shown in Eq. 2, linearly interpolated pixel values can be computed bymeans of 2-tap filtering using two pixels existing in each reduced layer(i.e., the closest two pixels positionally corresponding to theparticular pixel in the image data DD). Although a method for computingcomposite pixel values using linear interpolation is shown herein, it isalso possible to compute composite pixel values using splineinterpolation or other, higher-order method as the interpolation method.

The processing executed in the score value determining unit 35 will nowbe described. In order to selectively add motion blur to each pixel inone of either the horizontal direction or the vertical direction inaccordance with the parameters 1, the score value determining unit 35uses the previously input score values Ex and Ey as a basis forgenerating selection information specifying whether motion blur is to beadded in the horizontal direction or the vertical direction, as well asweighting information specifying the weights by which to multiply eachcomposite pixel value for given directions of motion blur.

The method whereby the score value determining unit 35 selects thedirection in which to add motion blur may involve comparing the inputscore values Ex and Ey. More specifically, the score value determiningunit 35 compares the magnitude of motion in both the horizontal andvertical directions, and then decides to add motion blur in thedirection of greater motion. In other words,

-   -   when Ex≧Ey: add horizontal blur; and    -   when Ex<Ey: add vertical blur.        On the basis of the above, the score value determining unit 35        outputs selection information specifying the direction in which        to add motion blur to the selector 34. In the present example,        the size of the information sent to the selector 34 may be one        bit for each pixel to which motion blur is to be added.

Another role of the score value determining unit 35 is to computeinformation on the weights by which to multiply each composite pixelvalue, and then output the computed information to the pixel weightingprocessor 36. The method for determining the weighting will behereinafter clarified by way of example in the detailed description ofthe pixel weighting processor 36.

The processing executed in the pixel weighting processor 36 will now bedescribed. The pixel weighting processor 36 receives a particular pixelLo(N) of the image data DD as input, together with respective compositepixels generated from each reduced image for one of either thehorizontal direction or the vertical direction.

Hereinafter, it will be assumed that three layers of reduced images forthe horizontal and vertical directions in accordance with the parameters1 are generated, and furthermore that the direction in which to addmotion blur is determined by the selector 34 to be the horizontaldirection. Herein, the composite pixels generated from the pixels ineach horizontally reduced image are expressed as L₁(N)′, L₂(N)′, andL₃(N)′.

At this point, output pixel values L_(out)(N) are similarly computed onthe basis of weighting information input from the score valuedetermining unit 35, wherein linear weighted addition of the above pixelvalues is conducted according to

L _(out)(N)=w ₀ *L ₀(N)+w ₁ *L ₁(N)′+w ₂ *L ₂(N)′+w ₃ *L₃(N)′  (Equation 3)

wherein w₀ to w₃ are the weights by which each pixel value ismultiplied. More generally, the above can be expressed as

$\begin{matrix}{{L_{out}(N)} = {\sum\limits_{K = 0}^{M}{w_{K}*{L_{K}(N)}^{\prime}}}} & \left( {{Equation}\mspace{14mu} 4} \right)\end{matrix}$

wherein M is the number of reduced images generated in a givendirection, and wherein L₀(N)=L₀(N)′.

From the input score values Ex and Ey, the score value determining unit35 computes weights w_(k) for each reduced image and with respect toeach pixel in the image data DD. The score value determining unit 35then provides the computed weights to the pixel weighting processor 36.

The generation of weights applied to each pixel value herein assumes theuse of the parameters 1. An exemplary method for determining the weightsw₀ to w₃ will now be given.

Given score values Ex and Ey positionally corresponding to a particularpixel, the larger of the component values is designated V. Subsequently,these component values of Ex and Ey are successively distributed on thebasis of the following weight determining rules.

Step 1: w₀=1; if V≦1, then set w₁=0, w₂=0, w₃=0, and go to step 5.

Step 2: if V−w₀>4−1, then set w₁=3. Otherwise, set w₁=V−w₀, w₂=0, w₃=0,and go to step 5.

Step 3: if V−(w₀+w₁)>16−4, then set w₂=12. Otherwise, set w₂=V−(w₀+w₁),w₃=0, and go to step 5.

Step 4: w₃=V−(w₀+w₁+w₂).

Step 5: normalize by dividing each value of w₀ to w₃ by V.

It should be appreciated that the above is merely one example, and thatthe present invention is not limited thereto. In particular, since adivision operation is produced in step 5, processing costs are increasedwhen realizing by means of hardware. For this reason, it is possible toreduce processing costs by defining respective weights w_(k) withrespect to V in table form.

FIG. 18 illustrates an example of such a weighting information table. Inthe table shown in FIG. 18, the first column indicates the score value E(Ex or Ey) for the selected direction, while the second to the fifthcolumns indicate weight values by which to multiply respective pixelsextracted from each reduced layer.

The numeric values given in FIG. 18 are merely given by way of exampleand are not limiting. However, smaller score values E preferably resultin larger weight values in upper layers (i.e., the base layer and thereduced layer 1), while in contrast, larger absolute vector valuespreferably result in larger weight values in lower layers (i.e., thereduced layers 2 and 3).

In this way, weighting information is generated in the score valuedetermining unit 35 and then supplied to the pixel weighting processor36. Subsequently, the pixel weighting processor 36 conducts the linearweighted addition given by Eq. 3, for example, and thereby computesoutput pixel values L_(out)(N). The pixel weighting processor 36 thusyields output image data OD wherein jerkiness has been reduced.

[2-4: Summary and Conclusions]

The foregoing thus describes a motion image imaging apparatus 200 inaccordance with the first embodiment of the present invention. Accordingto the above embodiment, motion images with reduced jerkiness thatappears more natural to the human eye is output.

More particularly, in the present embodiment, motion blur is added tounit images (image data for single frames, for example) in image data DDthat has been imaged and input. The motion blur is added on the basis ofa per-region still/motion determination, as well as imaging informationCI expressing parameters whereby the unit images were imaged. Thestill/motion determination in the motion region determining unit 40 isprocessing to determine, on a per-region basis, whether a given regionin a frame (i.e., a pixel unit or pixel block unit made up of aplurality of pixels) is a motion region or a still region. It ispossible to conduct the still/motion determination using simpleprocessing, such as by comparison with adjacent frames, for example.

Score values E that indicate the amount of motion blur to be added aregenerated by using the imaging information CI, which may include shutterspeed information, aperture value information, focal length information,and gyroscope rotation information (e.g., information indicatinghorizontal and vertical motion of the imaging apparatus), for example.

More specifically, score values E that cause a suitable amount of motionblur to be added on the basis of imaging information are generated formotion regions detected by the still/motion determination. In so doing,suitable amounts of motion blur can be added to regions in the motionimages where jerkiness is likely to occur.

Consequently, motion images with reduced jerkiness that appears morenatural to the human eye can be output without the heavy processing loadincurred by detecting motion vectors for each region within a frame.Moreover, since suitable motion blur can be simply added without theheavy processing load associated with methods such as motion vectordetection, apparatus simplification and cost reduction can also berealized.

Furthermore, the processing to add motion blur is primarily realized bymeans of spatial filtering, and since intermediate frames are notgenerated, the amount of memory used to store image data during theprocessing sequence can also be reduced.

The spatial filtering process outputs results by conducted weightedaddition of pixel values in each layer obtained as a result of executinglayered filtering. Consequently, the filters in each layer can berealized with small tap numbers. In addition, the filter tap number doesnot depend on the amount of motion blur to be added, and thus the use offixed tap numbers is also possible.

Realization of the embodiment by such recursive processing isparticularly suited to hardware-based processing apparatus. Obviously,however, it is possible to realize the above embodiment by means ofsoftware, or by a combination of software and hardware.

As described with reference to FIG. 2, the image data OD processed bythe motion blur adder 30 may be output by transmission from the transmitprocessor 60, or recorded onto the recording medium 90 by the recordingprocessor 70. Consequently, a playback apparatus can simply carry outexisting playback processing steps to play back either transmitted datatransmitted by the motion image imaging apparatus 200, or recorded datarecorded on the recording medium 90, without conducting processing toadd motion blur like that described in the present embodiment. In sodoing, such a playback apparatus is able to cause motion images withreduced jerkiness to be displayed on a display apparatus.

[3. Second Embodiment: Playback and Receiving Apparatus]

The configuration of a playback and receiving apparatus 300 inaccordance with a second embodiment of the present invention isillustrated in FIG. 19. The playback and receiving apparatus 300 is ableto receive image data sent by broadcast or other media, play back imagedata recorded onto a recording medium 91, and then display such imagedata.

A receive processor 110 receives incoming image data transmitted bymeans of television broadcast, for example. The receive processor 110may also be configured to communicate on a network and receive imagedata by downloading or other delivery format. The receive processor 110conducts receive and demodulation processing, and then supplies thereceived data stream to a motion image decoding processor 130 and acontroller 170.

A playback processor 120 performs operations to play back image datawith respect to the recording medium 91, and then supplies the resultingdata stream to the motion image decoding processor 130 and thecontroller 170.

Both the image data received by the receive processor 110 and the imagedata played back from the recording medium 91 is encoded image data thathas been superposed with imaging information. By way of example, it isassumed that information regarding the imaging apparatus parameters usedto acquire the image data is appended thereto as metadata. The imaginginformation metadata may include, for example, shutter speedinformation, aperture value information, focal length information, andgyroscope rotation information.

The controller 170 extracts such imaging information from the imagedata. Subsequently, the extracted imaging information CI is supplied toa motion region determining unit 150.

The motion image decoding processor expands the compression-encodedimage data supplied from the receive processor 110 or the playbackprocessor 120, and then supplies the decoded image data DD to the motionregion determining unit 150 and a motion blur adder 140.

The motion region determining unit 150 and the motion blur adder 140 aresimilar to the motion region determining unit 40 and the motion bluradder 30 in the first embodiment described earlier. In other words, themotion region determining unit 150 conducts still/motion determinationprocessing to determine, on a per-region basis, whether the regions inrespective frames are motion regions or still regions. In addition, themotion region determining unit 150 generates score values E indicatingthe amount of motion blur to be added by using the imaging informationCI (i.e., information such as shutter speed information, aperture valueinformation, focal length information, and gyroscope rotationinformation, for example).

On the basis of the score values E, the motion blur adder 140 addsmotion blur to the input image data DD, and subsequently outputs outputimage data OD wherein jerkiness has been reduced.

A image display unit 160 then outputs and displays the output image dataOD processed by the motion blur adder 140.

In the above playback and receiving apparatus 300 in accordance with thesecond embodiment, jerkiness is suitably reduced in image data that hasbeen received or played back from a recording medium, thereby realizinghigh-quality image display. Additionally, since motion blur can be addedsimilarly to the first embodiment and without the heavy processing loadof motion vector detection or similar methods, suitable motion blur canadded using simple processing, thereby enabling simplification of theplayback and receiving apparatus and reduction in costs.

The foregoing thus describes a first and a second embodiment of thepresent invention. However, it should be appreciated that the imageprocessing apparatus and image processing method of the presentinvention are not limited to the foregoing examples, and that variousmodifications are of course possible without departing from the scopeand spirit of the present invention.

Furthermore, although the foregoing embodiments describe a motion imageimaging apparatus 200 and a playback and receiving apparatus 300including the basic configuration illustrated in FIG. 1 as the imageprocessing apparatus 100, the image processing apparatus of the presentinvention can also be applied to a variety of devices that conduct imageprocessing. For example, it is feasible to apply embodiments of thepresent invention to an image communicating apparatus, an imagerecording apparatus, a game device, motion image editing equipment, or amobile phone, for example.

The image processing apparatus 100, the motion image imaging apparatus200, and the playback and receiving apparatus 300 described in theforegoing are configured to be logical assemblies of a plurality ofprocessing components. However, it should be appreciated that theprocessing components illustrated in the respective configurations aboveare not limited to being housed in a single physical unit.

[4. Recording Medium]

An embodiment of the present invention may also of course be conceivablyrealized by a recording medium that stores instructions for operationsequivalent to those of the foregoing image processing apparatus 100. Theinstructions for the operations may then be read from the recordingmedium and executed on a general-purpose computer or similar informationprocessing apparatus at the time of recording or playing back motionimages, for example.

More specifically, although the respective processing componentsconstituting the image processing apparatus 100 described earlier may berealized by means of hardware designed using an FPGA, for example, it isalso possible to cause a computer or other information processing deviceto read instructions from a recording medium and thereby execute theimage processing realized by the respective processing components.

For example, a recording medium may be provided storing a program in theform of image processing application software. The program causes acomputational processing apparatus to execute the operations of theabove motion region determining unit 2 (40, 150), the image acquirer 1that functions to acquire the imaging information CI, and the motionblur adder 3 (30, 140), for example. Thus, by providing such a recordingmedium, suitable image processing can be realized on a personal computeror similar apparatus.

More specifically, the program stored on the recording medium causes acomputational processing apparatus (such as a CPU) to execute thefollowing steps.

First, the program causes the computational processing apparatus to makea still/motion determination for each region of respective unit imagesconstituting motion image data.

The program then causes the computational processing apparatus toacquire imaging information expressing parameters whereby the motionimage data was generated as a result of imaging by an imaging apparatus.

The program then causes the computational processing apparatus to addmotion blur to the motion image data on the basis of the still/motiondetermination and the imaging information acquired in the acquiringstep.

By means of a recording medium storing such a program, image processingequivalent to that of an apparatus in accordance with an embodiment ofthe present invention can be executed on a personal computer, a mobilephone, a PDA (personal digital assistant), and various other informationprocessing apparatus that make use of image data.

The recording medium storing such a program may be an HDD housed in apersonal computer or similar device. The recording medium may also beROM or flash memory incorporated into a microcontroller that includes aCPU.

Alternatively, the recording medium may be a removable recording mediumthat temporarily or permanently stores the program. In this case, therecording medium may be a flexible disk, a CD-ROM (Compact DiscRead-Only Memory), an MO (magneto-optical) disk, a DVD, a Blu-ray disc,a magnetic disk, semiconductor memory, or a memory card, for example.Such removable recording media may then be provided as packagedsoftware.

In addition, the program may also be installed from a removablerecording medium onto a personal computer or similar device, ordownloaded from a download site via a network such as a LAN (local areanetwork) or the Internet.

The present application contains subject matter related to thatdisclosed in Japanese Priority Patent Application JP 2008-194876 filedin the Japan Patent Office on Jul. 29, 2008, the entire content of whichis hereby incorporated by reference.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

1. An image processing apparatus, comprising: still/motion determiningmeans for making a still/motion determination for each region ofrespective unit images constituting motion image data; and motion bluradding means for adding motion blur to the motion image data on thebasis of the still/motion determination and imaging informationexpressing parameters whereby the motion image data was generated as aresult of imaging by an imaging apparatus.
 2. The image processingapparatus according to claim 1, wherein the motion blur adding meansadds motion blur to regions of the respective unit images determined tobe motion regions by the still/motion determining means.
 3. The imageprocessing apparatus according to claim 2, wherein the still/motiondetermining means generates per-region score values on the basis of theimaging information and the result of a determination whether a givenregion is a still region or a motion region, and the motion blur addingmeans adds motion blur to respective unit images of the motion imagedata on the basis of the score values.
 4. The image processing apparatusaccording to claim 3, wherein the still/motion determining meansgenerates the score values by means of weighted addition using pluraltypes of imaging information.
 5. The image processing apparatusaccording to claim 1, wherein the motion blur adding means adds motionblur such that the amount of motion blur increases with faster shutterspeeds, as specified by shutter speed information included in theimaging information.
 6. The image processing apparatus according toclaim 1, wherein the motion blur adding means adds motion blur such thatthe amount of motion blur increases with increased aperture values, asspecified by aperture value information included in the imaginginformation.
 7. The image processing apparatus according to claim 1,wherein the motion blur adding means adds motion blur such that theamount of motion blur increases with larger focal lengths, as specifiedby focal length information included in the imaging information.
 8. Theimage processing apparatus according to claim 1, wherein the motion bluradding means adds motion blur such that the amount of motion blurincreases with increased gyroscope rotation, as specified by gyroscoperotation information included in the imaging information.
 9. The imageprocessing apparatus according to claim 1, wherein the motion bluradding means includes a reduced image generator configured to generate,from the unit images, one or more reduced images having differentresolutions, and a pixel compositing processor configured to extract,from the one or more reduced images, pixels positionally correspondingto a subject pixel in the unit images, and, on the basis of a valuebased on the results of the still/motion determination and the imaginginformation, add motion blur to the unit images by conducting weightedaddition of the subject pixel and the one or more extracted pixels thatwere extracted from the one or more reduced images.
 10. The imageprocessing apparatus according to claim 1, further comprising: imagingmeans for imaging an object to generate the motion image data; whereinthe motion image data obtained by the imaging means is input into thestill/motion determining means and the motion blur adding means.
 11. Theimage processing apparatus according to claim 1, further comprising:playback means for playing back motion image data by means of recordingmedium playback operations; wherein the motion image data played back bythe playback means is input into the still/motion determining means andthe motion blur adding means.
 12. The image processing apparatusaccording to claim 1, further comprising: receiving means for receivingmotion image data; wherein the motion image data received by thereceiving means is input into the still/motion determining means and themotion blur adding means.
 13. An image processing method, comprising thesteps of: making a still/motion determination for each region ofrespective unit images constituting motion image data; and adding motionblur to the motion image data on the basis of the still/motiondetermination and imaging information expressing parameters whereby themotion image data was generated by imaging.
 14. A recording mediumstoring a program that causes a computer to execute an image processingmethod, the image processing method comprising the steps of: making astill/motion determination for each region of respective unit imagesconstituting motion image data; and adding motion blur to the motionimage data on the basis of the still/motion determination and imaginginformation expressing parameters whereby the motion image data wasgenerated by imaging.
 15. An image processing apparatus, comprising: astill/motion determining unit configured to make a still/motiondetermination for each region of respective unit images constitutingmotion image data; and a motion blur adder configured to add motion blurto the motion image data on the basis of the still/motion determinationand imaging information expressing parameters whereby the motion imagedata was generated as a result of imaging by an imaging apparatus.